How have we evaluated software pattern application? A systematic mapping study of research design practices
نویسندگان
چکیده
Context: Software patterns encapsulate expert knowledge for constructing successful solutions to recurring problems. Although a large collection of software patterns is available in literature, empirical evidence on how well various patterns help in problem solving is limited and inconclusive. The context of these empirical findings is also not well understood, limiting applicability and generalizability of the findings. Objective: To characterize the research design of empirical studies exploring software pattern application involving human participants. Method: We conducted a systematic mapping study to identify and analyze 30 primary empirical studies on software pattern application, including 24 original studies and 6 replications. We characterize the research design in terms of the questions researchers have explored and the context of empirical research efforts. We also classify the studies in terms of measures used for evaluation, and threats to validity considered during study design and execution. Results: Use of software patterns in maintenance is the most commonly investigated theme, explored in 16 studies. Object-oriented design patterns are evaluated in 14 studies while 4 studies evaluate architectural patterns. We identified 10 different constructs with 31 associated measures used to evaluate software patterns. Measures for ‘efficiency’ and ‘usability’ are commonly used to evaluate the problem solving process. While measures for ‘completeness’, ‘correctness’ and ‘quality’ are commonly used to evaluate the final artifact. Overall, ‘time to complete a task’ is the most frequently used measure, employed in 15 studies to measure ‘efficiency’. For qualitative measures, studies do not report approaches for minimizing biases 27% of the time. Nine studies do not discuss any threats to validity. Conclusion: Subtle differences in study design and execution can limit comparison of findings. Establishing baselines for participants’ experience level, providing appropriate training, standardizing problem sets, and employing commonly used measures to evaluate performance can support replication and comparison of results across studies. 2015 Elsevier B.V. All rights reserved.
منابع مشابه
An empirical assessment of the software design pattern concept
Context: The publication of the milestone textbook on design patterns by the ‘Gang of Four’ (GoF ) in 1995, introduced a set of 23 design patterns that are largely concerned with improving the practices and products of software development. However, there has been no comprehensive assessment of the effectiveness of design patterns, nor is there any evidence about any claims and factors that are...
متن کاملRequirements Engineering Related Usability Techniques Adopted in Agile Development Processes
Context: Over the last decade there has been a growing interest in the integration of agile software development process (ASDP) and user-centred design (UCD). However, there are no papers that study which usability techniques related to requirements engineering are being adopted in the ASDP, and there are no formalized proposals for their adoption. Objective: Identify which techniques related t...
متن کاملClassification and comparison of architecture evolution reuse knowledge - a systematic review
– Context: Architecture-centric software evolution (ACSE) enables changes in system’s structure and behaviour while maintaining a global view of the software to address evolution-centric trade-offs. The existing research and practices for ACSE primarily focus on design-time evolution and runtime adaptations to accommodate changing requirements in existing architectures. Objectives: We aim to id...
متن کاملWhat Is Resilience and How Can It Be Nurtured? A Systematic Review of Empirical Literature on Organizational Resilience
Background Recent health system shocks such as the Ebola outbreak of 2014–2016 and the global financial crisis of 2008 have generated global health interest in the concept of resilience. The concept is however not new, and has been applied to other sectors for a longer period of time. We conducted a review of empirical literature from both the health and other sectors to synthesize evidence on ...
متن کاملExtracting reusable design decisions for UML-based domain-specific languages: A multi-method study
When developing domain-specific modeling languages (DSMLs), software engineers have to make a number of important design decisions on the DSML itself, or on the software-development process that is applied to develop the DSML. Thus, making well-informed design decisions is a critical factor in developing DSMLs. To support this decision-making process, the model-driven development community has ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Information & Software Technology
دوره 65 شماره
صفحات -
تاریخ انتشار 2015